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From late 1974 through 1975, validity data vrntm obtained far & 
battery of measures designed for eduaatlctially and economically dlsad-* 
vantaged Bttnority group trainees In w'ork-^ training (mMpowar) progranis. 
A longitudinal study sample of 391 males and 590 feniales were tested at 
entry into the ^portunities Indus trl all mtlon Ceiitar (OIC) progratti that 
provides ramadlal and vocational training for those who lack marketable 
ikilla. This sample^ drawn from OIC*b In 29 citlas throughout the U, S 
was followed up at program completion Cyieldlng an N of 304) and approxi- 
mataly 6 to 8 nK?nths after completion of training. For post^program uses 
a aampla of only 154 respondenta could ibe locatedj of whom 103 had obtalaed 
f ul l-» t Ime asip loyinen t # 

Parforniance criterion Information gathered at program completloin 
(short=term critarla) consisted of 30 v^atiables which ware factor anmlyzad 
separataly^ for males and famalas to produce auch orthogonal criterion 
dimensions as ''^Training Program AdjuJtirient j'- ''Moaeta^ Expectations 
"Effective Job Planning,** "Personal-Social Adjuatraent,** and "Vocatlotial 
Confidence*" Thlrty-'flve post-program (Loriger-tera) critarion dltnensions 
which had to be derivad for a sample of males and females eombinad 
(becausa of the sampla size available), ^Islded criterion dlTnensiong 
designated as: **Ovarall Employment Success*" "Blue Collar Job Success 5" 
"Job Seeking Effort," and "Social AdjustMtit mid Job Planning." 

The measures of the battery for i^hlch predictive validity was to 
be aacartalned deal with: (1) Praetlcal l^aaonlng skills (In the fora 
of diractlon-followlng in hypothetical Jo* settings), (2) Vocational 

V 
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Orientation (as Job knowledie , Job seeking skills , Job holding akllls, 
vocational aspirations plans and interests), and (3) Attltudlnal Ter^ 
csptlons (as sBlf-Bstem, atttttida tct^ard authority figures ^ motli/atlon 
to seek employment and wlllliignesg to defer present gratification for 
fucura gain). IWenty scares derived from that teat battery ware corre- 
laced with factor scores pbcained from the program completion and post^ 
program criterion dlTnensions- 

Analyses of the resuits indicated that i 

(a) The most valid measuiea for predicting the Prograni Completion 
(short-term) critgria were those dealing with Job Seeking 
Skills, Job Holdtng Skills^ &nd Job ^'nowledge* The highiest 
validities were obtaintd in relation to the "Txainlng Program 
Adjustment'' crttarlow dimension (r's ^ mid .20 to mid .30 -s). 
Practical Reasonine ta^tg sho^ somewhat lover validities, 
while minimal, but significant, levels are achieved for the 
Self -Esteem and Deferred Gratification scalea. 
(b) In general the various meaiures yielded better short-term 
predictive validity for females than for males. For the 
female sample^ L#vel of Vocational Plans and Aspirations 
measures resulted In negative validities against several of 
the Program Complitlon criterloii dlmenslona* rhCe seenis to 
indicatej somewhat paradoxically ^ that women ysith higher 
occupational goais and desires tend to achieve lower scores 
on the program completion dimensions, 
Cc) Relatively little predictiva vall^dity was foittid for the testa 
using the Post-Program (longer-term) criterion dimensions, 
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Results, however, are highly tenuotis bscause of data quality 
mi Inadequate sample mlm whlcih did not parmlt: yeparata 
analysis for tnale and f^iwale aubasniplei * The relatively 
few ilgnif leant validltias (r's In the *20-g) found were 
iot '-Job Seeking Skilla,^'" "Attitude Toward Authority," and 
'^Practical Reasoning'' ttata against a Grlterlon difflenaion 
of "Blue Collar Job Succeia" and a dlehptomoue criterlQn of 
Etnployment Ci*e,j Employed Full rime vb. Never Employed follow- 
ing OIC Training) * 
(d) The results obtala&d with this sample of OIC trainees are 
similar to thoae QbtalnmA In previous validation atudlee 
Involving younger school dropouts enrolled In a youth work- 
training program (Neighborhood Youth Coirps)i Mean test scores, 
. factor loading patterM tot the battery, criterion factors, 
and predictive vallditiiis are geiierally conalatent for the 
OIC and the NYC samplM. 
It vaa concluded/ f ram the ^i^Xlditlis achlevad ^ that a nuniber of 
the ineaBiirea are proimislng as gutdanca tools for ©aspower training programa 
and couiij* under appropriate cirQWW@taiiceS| serve as useful heurlatlc 
devices to shape curriculum dev^lopiaent or to evaluate behavioral change 
result lrk| from program partlcipatlQn. In addition, the factorially 
derivM criterion dimsnslona can mat only serve future teat validation 
needs ^ but also provide scalea lor aasesaing programatlc effectiveness. 
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Validation of Assessment >tea6\ires for Use With 
Dlsadvantagad Enrollees In Wovk^Tralnlng Programa 

Norman Frerterg 
Benjainln Shlfftoerg 

Educational Testing SGrvicc 

BACKGROTOD MP PW^QSE 

The scarcity of valid behavioral me^^urea nepdad to issei^ tr-alnee 
attltudltial j cognitive and social capabliitles ^ has constituted a ttajor 
daficloncy in the conduct of work--traiTiiiig (fflanpower) prograw (Backer, 
1972; Office of Economic Opportunity^ 1972), Attempts to mm available 
paper and pencil tests for guidance or pl&^mmmnt gervices in fcho^a oecu"- 
pationally-orlerited training iettlngs (ietvittg largely lalnorlty group 
enrolleei from culturally, economically mi adueatlonally depriviid back- 
groimds) have been open to serious challenge, Basic crlttclsitma levelled 
at the daslgn of formal (lie., published) Inst rumen t a have t«dad to 
focus on their inapproprlateness for low-verbal skill dlsadvwtagad 
respondents in terms of format ^ subject watteri reading level, li^ngth, 
linguistic style J method of presentation and interpretation of m^tm^- 
these daficienctes being seen as the outcow of teat developmiin=t proce- 
dures and noCTiatlve data samples aimed at middle class, majority giroup 
members eKposed to conventional educational curricula (Karp & Si|al, 
1965; Lennon, 1964; Society for the Faychologlcal Study of Saci^l lisues, 
1964) , 

In an attempt to counter most of the rto^tcomlngi cited ^ a tmt 
battery was developed— with support from cTie Eniployment and tt^lnlng 
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Administration, U. O^partment of Labor'^-tiu^t taps broad cons truces 
pertinent to the currlcyluTn coinponents and prof efliionally perceived 
guidance or information needs of work-tratni-ng programs (Freaberg, 1968; 
1970) p More suitabii da^ign features warfe Incorporated in that battery 
In the fortn of both picCorlal and verbal iitaMTtiil<s ^ relatively few items 
per measure; with all it^m stems and response alc«irnatives orally pre-- 
santed by the examiner in untlmed fashion i It^m content, verbal level 
and style were aimed specifically at adoleieant Qt young adult groups. 
MeaBurea of the battery dealt with areas ol CvalMe ability in the form 
of (1) practical reaaortiflg skills (designed abound Job simulated direction- 
following tasks) , (2) vocational knowledge m& swweness (as j Db-seeklng 
and Job-holding skills, Hnowledge of job requir^w^nts ^ vocational plans ^ 
aspirations and Intere^ti)* and (3) attltudlnal perceptions (dealing with 
constructs of self-esteem^ work motlvatiDn^ wtllif^gness to defer gratifi- 
cation and attitude toward authority figures). 

Data uaed for e^cmitiing the characterli ttca these meaiurea were 
obtained from samples of enrollees in a national TOrk-training program, 
serving niinorlty group school dropouts-=i ,e , ^ th% Neighborhood Youth 
Corps j Out^of' School PTO|ram (U,S. Departmant of l^abor, 1971), It was 
found that the tests poiaesa reasonable lewl.^ ot reliability for opera- 
tional purposas^-'with a tiu:n^er of them having potTOtially useful patterns 
of concurrent and predictive validity against A variety of performance 
criteria such as proflcltancy ratings by guidanca -counselors ^ training 
supervisors, peers and employerSs post-training, job success (e*g*. Job 
level, starting salary , raises) and comnunitys perianal and family adjust- 
ments (Freaberg, 1970 1 Fitaebarg Reilly, W72;; Fraaberg, 1974). Among 
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the most valid of the meaiUTfes were those that required som fora of 
cognitive capability (l,e,» practical reasoning, job knowledge and job- 
saaklng skills)* AttitudlnQi scales that showed lesser, but aignificant, 
levels of validity were the tttiea dealing with the trainee's feelings of 
personal worth (self-eBteetn) his attitude toward peraoTis In authDrltyj 
and his awareness of proper biihavlor in the work environment* 

The value of the instruments shown to date, has been dependent v 
almost exclusively, on their use with san^les of relatively youthfijl 
enron^es (approxlTnately 14 to 21 years of age) in a manpower progxain 
that provides only generalised aspects of ^*?ork eKperlence" as opposed 
to a program that concentratea on more specific vocational or technical 
skills training,* Thus, there remains a need to Investigate the wider 
applicability of the battery to those programs that Incorporate the 
development of vocational akiXla in their training curricula and that 
also tend to serve disadvantagad trainee grDUps of a wider age range. 

The present study attatjipta to address that need by applying the 
measures to samples of tralnwg of the Opportunities Industrialization 
Canter (OIC) manpower trainiwe program (OIC^s of America, 1975), a 
nationwide program offering v^madlal and occupational training to adults 
who lack the cultural, economic and technical capabilities needed to 
Qbtain and retain employment. In order to determine the suitability 
of the tests for use with such a population ^ it la desirable to examine; 
(a) the value of each of the tasta In the battery in terms of its levels 
of concurrent and predictive validity for a variety of perfofmance 

* See Levi tan and Mangum (19M) for a description of the various types 
of work-craining programs that have been federally supported* 
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criteria, (b) some contrasts between the results obtairiid aad tlioae of 
previous work-tralnitig program aamplee. In temii of getier^l rfleaauramenc 
characteristics (miafls ^ varlancei, fictor patC€ms, vaXidl-ty coefficients) 
and^ (c) the av^allabili-ty and applicability^ of performaic^ criterion 
meusures that am be used to define tnlriing *'^uccae8" and: tha cohirent 
outcoraa dlmenitons ttiac mdeilie and help e^plalti those mu^lti-f ace ted 
behaviors • 

A, Tha Teat Battery 

The choice of ineesu«Trtent constTUcts that comprlJi the baEtery ^ had 
originally been based on inforimtioti derived from wotk- tradnlug program 
professionals (e.g . j vocational coimselors, wrk-sit« i upe^vt aorSj job 
developersi trainirig project directors) regaTding trainee teeh^vl or^l 
capabilities deemed most: Importaiit In ^hapliii their trainir^g effcrti 
and thiis in the po tan tial emp loy ability and eve rail ^idjuaticien t of the 
trainee* These constructs were vsed to deflrie 13 aepirate tests that 
are moat conveniently groupsd mder thiree betiavloiffll catag^rles of (l> 
job-related practical rfeaionijii skill, (2) voca tlonal orlen^taclon, and 
(3) attltudinal Tsrcaptions of self and others, rhe meaaui^is are de- 
scribad briefly ^elot^ * 

1- Pyactlcal Raas oailng Skill la ccmprlsed of three teste consisting 
of: 

• Practical Reasoning- Zip Coding CIO it^ms)^ prcvldea ttie 
respondest with inforTnatlon needed for a h^'pothe tlcal post 
office job re qui ring him to sor t oiaiZ ufling sip codes* 
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<)usgtions are presented In a niultlpla-^chDlGi foimat pertaining 
to that task. 

• Practical ReagQ nin ^ - Hap Madlng (10 Icains) i provides a map 
llluitratlng ieveral square city blocki ilsng with InfoTOatton 
needed to ansvsr miiltlple choice quest Ions about delivering 
naterlali to different locations show on the map, 

• Practica l Reasoni ng - File and S orting CIO items) i requires 
th€ reapondent to sort out potential Job appllcmtg based oa 
their background as presented on a let of numbered file cards, 
Nultiple-cholc^e Items invQlve matching file card Information 
to job requtreD^ats stipulated in each Itas atemi (This * 
measure is used exclueivBly vtth oales j since prior results 
had indicated that the task lacked sufficient difficulty for 
fenalea and minimal variability*) 

Vocational Orientation Is a category comprisltig six nteasuris i 

• Job Knowledge (27 Items) : requlrsa answers to multlple^cholce 
Items regarding a variety of jobs (e^g,, carpenter ^ auto 
machanle , pDllcemati) In teraa of their educational requlrsraents ^ 
starting salary ^ primary task performed, hours of worki place 
Inhere work Is performed antf tools uttlliedi 

• Job Seeking Skills (17 items) i presents niultiple-cholce Items 
dealing with ^ays of looking for j obs ^ some of which entail 
InterpreCaticn of newspapar ^ant ade and pertlons of job 
application blanks « 

■ Job Holding Skills (11 items) t depicts situations that roight 
be encountered on a job vtth regard to supervisors' requestSp 
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appropriate dTess, punctuality, etc., for wilch the reapoiiderit 
chooses the reply he would give, or the most appropriate 
bahavlor called Sor^ in that situation. 

• Vocational Agplratlons (16 Items)- utiLlEis itemi that present 
a wide range of jobs both plctorlally md by Job tttle (l^e,, 
from "Laborar" to "Scientist") . ae respondent indicates tha 
degree to which he wuld aspire (prefeT) to engage in each 
occupation shown. 

• Vocational Plana (16 items) i Incorporates itama shelving the 
same jobs as the Vocational Aspiration boolcLaC with response 
alternatives that request the degree to which the respondent 
actually plans Clntends) to enter such an occupation, (Both 
the aspirations and plans measures are scored leparateiy along 
with a differenca score obtained by subtracting plans from 
aspirations * 

' Vocational Interest (28 itenii)i present.^ Dictcrlal and verbal 
Information regaTdlng job tasks characteristic of a number of 
occupations. The Tespondent Indicates the degree to which he 
would favor performiiig each taskp Separate scale scoris are 
derived for each of seven interest categories (4 Items each) 
designated as Clerical, Service, Technical^ Outdoor, Sclenca^ 
Business, and Aesthetic* 

Attltudlnal Perceptions are defined by the following four tests; 

* Attitude Toward Authority (12 items) i In which scenes depict- 
ing responses of an adolescent to authority figures (e.g., 
teacher, policeman, parents, judge, etc.) are used to elicit 
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the degree of pro- or ancl^a^thorlty feelln|fi hy tht rgepen- 

* Self -Eg teem (16 ttams) i depicts situations ptctorlally In 
y?hich the reapondent Indicates the degree to ^hleh he feels 
himself 'Worthy" of the deslralle outcomes (getttng a diploma 
j obp promotion i ete.) i 

• MotivatlQn for VocatlonaL Achievenient (17 ltena)i presents 
statemeatei aa Item st^mm^ that lear on thm desirability^ of 
seeking and malatalming emplaymeiit and wilLlngneas to suffer 
inconvenience In aeceptliig eBtploysent* These require the 
liidividual to respond (LLkert-scali format) In terms of the 
degree of agreement with the statenieiit or tie degree to ^hlch 
a Job Is acceptable In spite of ^desirable features* 

;• * ^ferred Gratif ication (16 items) i praeenta statements 

dealing ^Ith the willlngneas to delay preaeiit reward for 
future gain to \^hlch the respondent IndlcatCB* his degree of 
agreement on a five^poixit scale (Llkert-scale format)* 
The itema of each teat appear i% itnall (4" x 8") tooklats with a 
single item to a page, They are intended for oral administration t© 
small groups of r.raineea (nasclmutQ of about 12) and are wtlnedi being 
paced essentially by the exanainer ^ho reads all item stems and response 
altematlvag. Qf Che 13 measureSp 5 are wholly verbal In content and 8 
contain both pictorial and verbal Information- Whmtm pictorial icenea 
are prasented, a young adult is alwys depicted as a central flgtire, 
thus necessitatliig separate forM of those S booklets for male and 
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female respondents. More complete descriptions of test ritlonale and 
itam content are found tn FreebeTg (1958, 1970) , 

2* Perform ance Crltsrion Heasurag 

Two sets of trainee perforaance outconies are to te used as criterion 
inaasures for predictive wlldation of the test battiry: 

1* fro grain Completion (ahort-^term) Criteria - which were obtained 
at the time the trainee complited the QIC skills tralnlnj pro- 
grani. Tlieie were obtained frotn a quaationnaiafe that yielded a 
tptal of 30 usabla variables. These are listed in Appendix A 
with some €3^ lanatory 'information. The items had been derived 
from five a priori (rattcnal) perfDrmanca categories of <a) 
Vocational A\^areness md Planning Skills Ce,g., a;^are^&ss of 
important aspect;s of JobSj knowledge of steps to tike to find 
emplpyment^ leirel of jobs planned f or) * Cb) TersonaL-Social 
Adjustment (e.g, , In regard to family, coimunlty ^ f olice) , (c) 
Vocational Motlv^ation (e *g* ^ wiliingneaa to accept traliitngj 
general views of desirability of employinant) , (d) Jralnlag Pro* 
gram Adjuitment (e,g, ^ afaiencigp nuntar of course changes, 
Judgments of proficiency by others^ i,e,, eounselori , ins true- 
tors, peers) ^ and (e) Vocational Self-Confidence (e,g., per- 
ceptions of ability to "maka it" In a job setting)* 
Post-PrograTO (longer-term) Criteria - consist of 35 variablas '^?^ 
(described in Appendix B) obtained 6 to 8 months after OIC 
training prograin completlcn. Items that coBiprisa tlie question- 
naire were derived from four rational dimensions, or categories, 

17 

-8- 



of loiiger-terin oijtcoies dtfliied ftsi (a) Extent and Levc*i nf 
Etnployment (t.g,, quality of job qbtalnedp length of job stay, 
salary, nunter of jobs h^ld) ^ (h) Job Petformance md Adjust- 
ment (e.g. 5 jab satlsfacitloti , salcinr raises, promotionSi aalary 
expectations , employer proficiency rating) (c) iPerional-Social 
MjuBtmeat Ce^g,^ fmLLy relatlotiai coMunity adjustment, hMlth, 
use of credit, aavlng money), md (d) job KDttvatlon and Planning 
(e.g., effort In abtalnlng Interview md filing applications, 
actions to take if pras^nt job li lost, attendanea at aehooli 
ninnber of sources used to find job, lev^el of future job plana) . 
Both the broad catagoriea and the variables chosen to define them 
were drawn from work- trailing program objeatlves outlined in inabllng 
manpower legislation (Eoanoaile Opportunity Act, 1966; Federal Register, 
1969), federal agency standards (U.S * Pepartment of Labor, 1971), and 
rssearch llteTature of the aort davoted to Isiuas la work-training pro-- 
graBi evaluation CBorus If Tash, 1970; l^olay, et al,, 1970; National 
League of Cities, 1973). 

The criterlo:n queatlom aires for trainees were designed for oral 
administration on an Indlvidtial basis in a 20 to 30 minute session. 
Prior application of thaae criterion InatriMints wltb cross^sectional 
samples of Nalghborhaod Youth Corps traltieii and forroer tralneas (6 
TOonths to 1 1/2 years out of the program) had demonstrated the faaii'- 
bilLty of mosc of the variables for their Intended put^oiei (Ireeberg & 
Rellly, 1972). Certain modifications la scoring re required, however, 
f03r the present study because of distributional properties of some of the 
meaaurBs. For example , "nuiftbir ,of pay ralies'* and "nusiber of job 
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protnotionsp'' althDugh iomd to be suitable criteria in previous croae- 
secclorial $m4y , were found to be so highly skewed In this sample (few 
persoiis racel^lTig even a single pay raise or promotion) that they could 
only be applied as dichotontous variables (''Pay Raise va^ No Fay Raise" 
and '^Proniotton ^s. No Promotion'*), Apparently m poac-program slK-month 
folLow-irp parlod—lmpostd by smdy eoiistrainta'--dois not allo^^ sufficient 
tifflfe for mori thmii a amall proportioii of the ex--tMlnees to obtain pro-- 
rnoclons or pay raises; thua tindlng to weaken thely effectiveness as 
criterion measurij, 

C* Samplag and Data Collecei^ti 

Tlie test bittery was adtotntsteTed at program entry to a sample of 

391 male and 590 female tralneas enrolled in OrC projects located in 29 

cities throughout the S,^ Of the 981 trainees tested at entry, there 

were 304 (115 siales, 189 fettiales) who were able to he follcwed-up at 

"cofflplfitlon" of training for aimlnlstration of the Program Gosipletlon 
2 

Quastionnaire. Of the 304 traiaees there were 157 who could be located 
about 6 to 8 months after completing training and %mo were billing to 
fespDnd to the Pest-Progtam CJueatlonnalre , Of that 157, 103 C66X) had 
found full'tim eniployment--30 houTB per week for m least one week— 
after they left QIC, It is this group of 103 that conatltutes the basic 
saiQple for analyses of post-program criteria, sinm Che jcb-telated per-- 
foraaiice criterion measurea cm only be applied to those who hold 

See ^TpendlK C for lint of cities in which participating OIC^s are 

2 

TUmm to completion vairitd sotnavhat as a function of the particular 
vocational curricula in which the trainee was enrolled. In moat QIC 
prdjacts, howe^fer, the WrollMnt period la notnlnaily 4 to 6 months. 
Any trainee who remained beyond four months. was conslderid eligible 
for inclusion it\ the frograw CoDipleClon Sample, 
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employflJent* (Of Interest is thm fact that the 34% who regained ittiefiaployed 
rapresfint a proportion consonant with D, S* Department of Labor unemploy^ 
ment rates reported for teen-age and yoioig adult oilnority group members 
during th€ 1974-75 recession,) 

Eighty-t^o percent of the tralnaes followed up were elassiflabla as 
minority group members (65% BlaClCp 171 Hlspaiiic) and 181 as White, The 
age rwgi w^as from 17 to 35 with a rotan age of 20.7. Trainaea completed 
an average of 10.8 years of school and almost 60% indicated that thay 
were high schael graduates, mean tisie spent enrolled in OIC M# 

4*6 months. 

All phases of testing and questionnatre date collection were carried 
out by a daiilgTiated staff member at eaeh OIC project who was also a full- 
time OIC amployee - Each of theie individuals Cock part In a two^day 
training session that provided lamiliatlEatloii with the study purposes i 
content of the tascs and questloiinalresj proceduafes for tralnae follow- 
up , and instructlona for adminlatratloa of the materials* Responsibility 
for coordtnation of training and data gatheriiig was assumed by a profei- 
sional staff niembar of OIC in Phtladelphla, 

D. Dascrlp tlon of t he QIC Program 

In order to appreciate the nature of thm data collected during the 
course of this project is is deiirable to tmderstand the organisational 
charactarlatlcs of the Opportunities Indus trial Izatlon Center (OIC) 
prograia and its node of operation* 

ore was originally foimded in Philadelphia In 1964 with the goal of 
providing job training * primarily fot* Blacks and other minorities from 

*11- 



the inner city who lacked marketable skills. In addition to providing 
skills training for a wide range of occiipatlons , the OIC progrMTn placed 
heavy emphasis on aBslstlng trainees In their personal growth and devolap- 
ment. Virtually all OIC prograins require chat trainees go through a fsur-^ 
to sl3c-week "Feeder Program" which eridea^ors to liistlll appreciation for 
one's personal capabilities and a 'iucces^ orientation,*' along with 
Inatriictlofi in verbal and arithinettc skills, need^ for functioning In a 
vaxiaty of tha vocational curricula offered, Foiaoying the Feeder Program^ 
the trainee chooses a specific occupational skill area which he enters 
for an instructional period of about four to six raonths. These skill 
areas ramge frotn conventional trades (e,g,, machine shop, auto mechMlca) 
to clerical and health-related occupatlDns (eig. , practical nursing). 

rralnaas tend to be young adults, although Individuals over 30 are 
also accepted. Some form of stipend is paid to the trainees in most pro- 
graiina with the amomit dependent upon the ternis govirnlng the CETA grmt 
to the local contracting agency* In addition to remedial educational 
and vocational training, the program provides guidance counseling, job 
placement, and follow-up of trainees for up to alx months after com- 
plition of training. 

Between 1964 and 1973, OIC eicpanded from a single center in Phila- 
delphia to more than 100 centers in 43 states. During this period, the 
national organization (OIC*s of ^erlca) rsc^ived Its funds directly 
from the U, S. DepartMnt of Labor under a prime contract. With the 
passage of the Comprehensive Employment and draining Act (CETA), funds 
for OIC centers were channeled through local prime sponsors. This change 
In the fiscal and adminlstratlva relationship tended to loosen the strong 
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admitilstrative ties that hmd SoTmrly iaclated betwein the nattonal OIC 
orgatilEatlon and the local OIC ciiitixs* Kowsvtr, the natlDnal OIC con- 
tinues to provide cQtigultafcion and eertalii adinlnifltrativi sarvlMa, An 
Association of OIC EKSCUtiw Dlrictors also strves to Mlntaln comimi- 
cation among centers and to «lriforce OIC 's coinmitment to the slogan 
"We Help Ourselves*" 

E, Special Sampling Problfeittg^ 

A numbar of conatralnta ^countered in data gatherliig for the longi-- 
tudinal aamples should be clarified ^ since thty can effect sample blaaas , 
Che quality of the study r#gult8 , and the permissible generalizations. 
Most of these sampling problems have been found to be generic In any 
attempts to obtain research or evalmtlon Information from (or about) 
poverty-level p minority group trainees residing In Inner-eity areas, 
who have been characterized as so«called **hard-to-lpcate" papulations 
(Barnes^ 1972; U, S. Departwnt of labor , 1969), Special difficulttee 
have been cited in: (a) attimpts to locate respondents through conven- 
tional Inatitutional channals Cewployerst schools^ social organizations, 
etc) j (b) overconiing minority eonimunlty suspicion of ''outsiders" aeaking 
information, (c) the unwillingness of former tralneea Caapeaially those 
who have been out of a trali^lng program for a period of sonths) to 
cooperate as respondents to que sclorin aire aurveyss (d) a high degree 
of sensitivity to ''invaiion of privacy** by all ^inforffiatlon sources, and 
(e) recruiting profesaipnal intervleyeri lufflclently familiar ^tth the 
minority comnunity and the mhm locale In ^^hich they are to fwictlon. 
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Although the present invesfcigatlon would have been ©Kpected to 
encoijwter many of these same cor^qems, evsi^y effort was made in the 
itudy ^aslgn to minimize them, For example i the use of QIC staff 
personnel to administer the tait$ md to collect criterion data should 
have iwvtd to overconia some of the suspicion of outsiders that might 
other^lie have been generated. What could not have been anticipated, 
hommtp was the devastating mtf^ttm of the economic recession of 
1974-73— during the very time chat the project stiff was endeavoring 
to collect criterion data. The feoonomlc plrich caused a number of QIC 
lnatali4tlons to curtail their programs drastically and, In a few 
InstaiiefeSj to cease operations nXtogethert A nuinber of the OIC personnel 
who h#d been assigned to assist with the project and who had undergone 
orientation and training In data Mllectlon procedures were reassipied 
to ochrr dutiea s causing a subatAWtial;^loss of follow-up data* The 
Impact on trainees was also con^fderab;;a* Hany of them dropped out of 
the training program because th^y wuld not find part-time jobs to sup- 
plement their CETA training atlpi^nd. Others left training when they 
became discouraged about the job outlook. T^ua^ the severe attrition 
in the sample (from 981 down to 157) is a function of unanticipated pro= 
gram terminations j reassignment OIC pexsonnet, a higher than expected 
dropout rate among trainees, and m imwilltngness of some trainees to 
complete crlteTton questionnaire^ , 

F. Dgktf Analyses 

Glwn thfe 13 measures of thfe tmt battery alorig with the 30 program 
completion and 35 post-program CTiterlon measures, the analyses to be 
undercakM consist of: 
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(1) Examination of thm Gharacttrl^tlas of the tests md fcht total 
bmttety In terma of m^mm md v«iitficei , test Intercovtalation 
md resulting faetor pattama* fheae also allow for gMas 
comparisons between thm present #tudy sa^le md prsvioio^ 
results obtained with smples of youth wo rk-*t raining progiean 
inrollees, 

(2) Identification of major Inteipretabla ariterion diwiii^ioiri^ 
for use in validation of the battery^ by iKtraetlott of f^-ctore 
froa the 30 x 30 program coinpletion and 35 x 35 pQit^proi;?:^ 
criterion Intercorrelatlen matriofts and eoBputation,^ Ctow the 
loadings J of criterlpn factor scoMi.* 

03) Computation of validity eaefiAqlwts bet^^een each w#TOM^re of 
the teat battery md the factor ieorea for each of thm Wo 
sets of criterion dimenilona which constitutes the pximt^ 
analysis of the atudy* 
It ahould be pointed out that mliilng data ocourred at mmh fltage 
of longitudinal data collection, so that wlsaing^data analyssii mm 
required throughout (e,g.j in many Instan'aes respondents ahoB^ not to 
aaswer particular Items of the tests or quniitlonnaireai In othm ioiatan- 
ces ratings by peers ^ instructors, comiselovs » or mmployBTm mm 
avatlabla for significant proportions of thm sai^les.) Furtb^irwo:^^!, 
attrition for the post--progrM saaple, th^t reaulted In an N e*f 103 
esiployed individuals, necessitated factor iwialyses of the 35 ^tttmxlM 

*Factor scores ware conqjuted by post--multip lying each matrlK of % eciorea 
by the rotated factor loading aatrix, obtained frott a principal cotor 
ponents golution. 
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for th€ seKes combined ^ The program eorapl#ti«a ^ampla ilse of 304 (189 

ftmales and 115 mlmB)^ howavtr, prevldas mmm Tmuonmblm Justlficatiori 

for computing criterion 4toi«naloni separately by mx. 

As a smeondary mA hrimX analysis. It was al^a poislblt to ixamint 

changae in pre- and poit-test aeores for a mmU of 107 (71 fimales and 
36 nalas) frott the 981 trainees originally tert^d*, Fre-teBt scoras were 
those obtained at the tXmm of ent^ into OIC mA pOBt^teat scores at the 
time of completion of thu four to six week *^fmAm Program,'' dwing which 
eoncentrated Instructtom is given in remedial #ktWs, This relatively 
small subs ample was ohtnined on an OTsaleaWd,, voluntary basis by data 
collectors at a few OIC ^ites, who were wiUliig aKpand time and effort 
beyond the minlmim data gathering requlraTOWt$. M aueh| there are 
likely to be a number of unteiown selective hlmm in the subsaaple and 
the findings will be tPMched on only briefly in' fchi Results section below, 

RESULTS 

A* Th% Test Battary 

Means and stmdard d^Mlatlons for 12 tmm mi 7 Interest subscales 
are shown In Table 1 for fcha OIC sa^le of B$l m4 tot a sample of 560 
Nalghborhopd Youth Corp« tN.Y.C.) trainees phmim^. In 1972. 

In general the mem^ and variances are fairJiy eitnllar for the two 
samples, although, notably^ the mean scorea-^^^ith the exception of Level 
of Vocational Aspirations (LVA)— tend to be ilightly but consistently 
higher for the OIC sas^lite, of the nean dlffewncaa for the 12 testa 

and for three of the InCi^^st subscales do, hwwfev% reach significant 
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Table 1 

Test Meang mi SCradsrd Deviations for 
QIC N,Y,C. Trainees 







N Y r 
(N - 


560) 


01 C Sample 
(N «• 981) 




Test 


M 


g 


M 


g 


1, 


Job Knowledge 


19.4 


4.2 


19,9 


3.4 


2. 


Self-Esteem 


36.5 


4,1 


37,8 


3.9 


3, 


Deferred Gratification 


51,7 


6.9 


53.5 


6.8 


4, 


Level of Voc. Aspir, 


184.8 


35,1 


199,8 


32.3 


5, 


Level of Voc. Plans 


160,4 


34.2 


158,0 


36.3. 


6. 


Attit. toward Authority 


4a.4 


8,7 


44.7 


8.8 


7. 


Job Seeking Skills 


10.9 


3.4 


A* * w 




8. 


Job HQldlng Skills 


28.2 


4.1 






9* 


1 

Motlv. Voc. Achiavement 










10. 


PR-Hap Reading 


5,4 


2.6 






11. 


PR-Zip Coding 


7,0 


2,7 






12. 


PR-Flle Card 


7,7 


2,7 






rncarest i 










13, 


Aesthetic 


9.9 


2 .5 


10,7 


2,5 


14. 


Scienca 


9.5 


2 ,7 


10.6 


2.7* 


15. 


Technical 


8.8 


3,1 


9.2 


3.2 


16. 


Clerical 


10.9 


2,6 


11.8 


2,4* 


17. 


Businass 


10.1 


2 ,5 


10.8 


2.6 


18. 


Service 


U.l 


2.4 


11.9 


2.3* 


19. 


Outdoor 


6.9 


2.8 


7.6 


2.9 



* Mean diffarenca slgnifican.t mt th© .01 level 

i Not comparable measuifes In b'ath iaraples. Seven additional i teres ^are 
added to the original ten Hmm MVA meaaurt for use in the present study. 

*17" 
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levels of mean difference (p < .01). The OIC trainees are apparently 
more adept at Interpreting job want ads and application blanks (Job Seek- 
ing Skills) and more sensitive to appropriate conduct in a job setting 
(Job Holding Skills), than the younger, teen-age school dropouts of an 
N.Y,C. program, Most of the job tasks depicted for the three signifi- 
cantly higher Interest scales (Clerical, Service and Science) represent 
a number of areas in which OIC trainees are likely to be qualified by 
virtue of their training. 

As a means of eKaminlng the dimensional characteristics of the test 
battery when applied to the OIC population and, at the same time, provid- 
ing another level of comparison with prior findings, the test battery was 
factor analyzed. Intercorrelations of 20 scores for males and 19 scores 
for females derived from the teats and the Interest subscales are pre^ 
sented in TaMe 2 for both sexes* 

The patterns of zero-order r^s are seen to be broadly similar for 
both sexes~a similarity that is clearly reflected in the factor struc- 
ture of the battery. Factors were axtracted from each matrix using a 
principal components solution with varlmax rotation to orthogonality 
(Kaiser, 1958). Three readily Interpretable factors were found for each 
sex and are summarized in Table 3 on the basis of loadings of sufficient 
magnitude to enter into the interpretation (i,e,, approximately .30 or 
greater). Factor I ( Intellectual Ability ) shows virtually an identical 
pattern for males and females in its positive loadings on the Job Knowl« 
edge, Practical Reasoning, and Job Seeking Skills measures. Almost 
equally identical is Factor III, which is designated as Vocational 
^^^^^Q" the basis of its high positive iQadlngs on the Vocational 
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Tatls 2 

Intercorriktlons sf ^lom kt Itit BaUery 
(Haleg !< 3P1, Fiiales > m)^ 







-L 




X 


i. 




X 




















VfieaEisnal Incereit 










J__ 


X 


X 


i 


U 


12 




ii 


M 


16 


17 


18 


19 


20 


li 


Job Knowledge 




.343 


i23 


.04 


-,09 


.20 


.19 


.43 


,31 


.05 


.23 


,27 


.33 


.06 


.16 


,09 


,01 


,03 


,17 


,07 


L 


SelHsEieiB 


,04 




.32 


.16 


,09 


,08 


M 


.23 


■ \^ 


,15 


,13 


.25 


,27 


.19 


.26 


.22 


.19 


.30 


.30 


.10 


3i 


Itefirred GtaEifiestisi 


.18 


,16 




.04 


-.03 


45 


,30 


,22 


.36 


.11 


,13 


.15 


.18 


.03 


,10 


.07 


.12 


,13 


.09 


-.02 


4. 


Vocae, AapiraEioni 




,08 


.06 




,48 


,43 


.07 


-.09 


.21 


.12 


-.04 


.00 


.10 


.45 


.43 


.39 


,43 


,40 


.44 


.36 


S, 


VociC, FUni 


-,19 


.03 


-.02 


,so 




•,J9 


-.01 


-,19 


.01 


,14 


-.18 


-,10 


-,08 


.25 


,33 


,28 


' ,34 


,32 


.30 


.32 


£ 

Pi 


Asplr, ainifi llans 


,16 


,04 


,03 


.3S 


-.60 




.13 


.13 


.11 


-.02 


.15 


,12 


.17 


.16 


,11 


.09 


.06 


.06 


.12 


,02 


7. 


Ak. toward AuthorUy 


,21 


.18 


.31 


.09 


-.04 


,13 




.30 


,17 


.04 


.20 


.21 


.16 


-,04 


,04 


.07 


,04 


.07 


.02 


,04 


s. 


M Seeking Iklllg 


.46 


.03 


,20 


-,0S 


-.15 


.10 


.26 




.34 


.10 


.43 


.47 


.44 


.02 


.05 


.01 


-.05 


.03 


.08 


.03 


9t 


Job Holding iklUi 


.30 


.16 


,22 


,10 


-,os 


.16 


.23 


.25 




.16 


.20 


,28 


.30 


,16 


.23 


.14 


,15 


.21 


.26 


.10 


10, 


ttotlv.for Voe. Aehiev, 


,07 


.07 


,19 


,oa 


.05 


.04 


.13 


.16 


,14 




-.03 


.03 


.04 


.06 


,07 


.11 


.09 


.13 


.10 


.16 


1I« 


PR.H3P Hading 


.u 


.04 


,08 


-.09 


-.20 


,11 


,19 


.42 


,13 


.05 




,13 


,44 


-,04 


-,02 


-,05 


-.14 


-.07 


.01 


.04 


12. 


FMip CodiDg 


.39 


,05 


.09 


-.03 


-.20 


,16 


.21 


,51 


.17 


,06 


.49 




.60 


.0? 


.07 


.00 


-.01 


,14 


.09 


.04 


Ih 


n-nu cird^ 




























.04 


,03 


.03 


,08 


,08 


,10 


.03 






-,18 


.09 


-.03 


.38 


.37 


-.03 


.01 


-.10 


-.03 


,08 


,00 


-.07 








.41 


,47 


.48 


.53 


.35 


U, 


Selenet 


-.09 


.OS 


,00 


.42 


,29 


.09 


.07 


-.03 


,07 


.OS 


.01 


-,04 




,34 










.51 




16. 


fechQleil 


-,U 


^.06 


-,09 


,37 


.37 


-,04 


-,03 


-,19 


-.05 


.04 


-.10 


-.15 




.46 


.49 




.36 


,46 


,41 


.52 


17, 


CUrleal 


.01 


,08 


.03 


,29 


.27 


-.01 


,11 


,04 


.12 


.11 


-.07 


.00 




,19 


,31 


.18 




,61 


,46 


.29 


18, 


luaiDeii 


-.14 


.06 


,05 


.44 


.39 


.01 


.06 


-.09 


,04 


.08 


-.07 


-.06 




.47 


.39 


.40 


.52 




.41 


.31 


19, 


SiEVlce 


-.07 


,03 


-.02 


,46 


,3? 


,04 


-,05 


,11 


.08 


,00 


-,06 


-.06 




.42 


*42 


,37 


.29 


.43 




,46 


20, 


Oyedoor 


-.06 


.01 


,03 


,22 


,18 


.02 


-,04 


-,02 


.03 


.06 


,14 


.03 




.3? 


.37 


.33 


.06 


,24 


.25 





r'l for iiiii above iU%mil\ tmlu bilow 
Tilt uiid vlth sales @Qly 

r'a of di or griilif ilplfleasi at the .01 Igvel for mlu 
t's of .11 @f %mm liplfioal ai ih@ ,01 U^l far fiiilei 
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Table 3 



Factor I 
j gopltlve Skill 



M iDadlng 

PR*2ip Goding .81 
PR-!tap Riading J9 
PR-Fili Cards ,74 
Job Sitkini Skills ,70 
Job Kngtfledgi ,44 



Factor Variance = 2,8 



I 

N 
0 

I 



Factor I 
Copitlve Skill 



M Loading 

PR-Zip Coding .79 

Job Seiking Skills .76 

PR-Hap Riading ,75 

Job Knowlidge M 



Factor Varlimce s 2,5 

)0 



ated Factor Loadlnp for Test Batttry 
(Hales N s 391) 

Factor TI 
Positive Attitude 



Test Loadln; 

Deferred Gratification ,74 

Self-Esteem .66 

Job Holding Skills ,63 

Attit. Toward Authority JO 

Job Knowled|i ,39 

Hotiv, for Achievifflsnt .36 

Job Seiking Skills ,30 



Factor Variance § 2.2 



(Feialis, N ^ 390) 

Factor 11 
Fosltive Attitude 



Teat Loadini 

Deferred Gratlflcatiofl .72 

Attit.Toward Authority .59 

Motiv, for AchleviMit ,53 

Self-Esteei , ,52 

Job Holding Skills .45 



Factor Variance ^ 1,7 



Factor III 
Vocational Mltion 



Test 


Loading 


Voclnt.-Sclenci 


.79 


Voc.Int, -Aesthetic 


.74 


Level of Vocat. Aspir. 


.73 


Voc.Int, -Service 


.73 


Voc, Int. -Business 


.71 ■ 


Voc.Int. -Technical 


.70 


Voc. Int. -Clerical 


.69 


Voclnt .-Outdoor 


.63 


Level of Vocat. Plans 


.46 



Factor Vari^ce ^4.5 



Factor III 
Vocational Mition 



Test 


Loading 


Voclnt, -Clerical 


.74 


Voc.Int. -Business 


.72 


Levil of Vocat. Aspir. 


.70 


Voclnt, -Service 


,65 


Level of Vocat . Plans 


JO 


Voclnt. -Science 


.48 


Voclnt. -Aesthetic 


.39 


Voclnt, -Technical 


.30 
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Factcr Varlmce ^ 2.8 
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In teres t scales, along with Level of Vecational Aspirations and Livel 
of Vocational Plans. The only minor ssk difference on this factor is 
a reasonable one, in that females who score high on the dimension do not 
tend to favor the choice of job tasks depicted on the Outdoor scale; 
probably because they cannot perceive these as appropriate for themselves 
aa women living in an urban environment (i,e., tasks of sowing crops » 
caring for farm animals) , 

Factor II is similar for both sexes as one that unequivocally 
defines the trainee holding Positive AttltudeB by virtue of having its 
hlgheat loadings on those measurea that deal with Self-Esteem, Deferred 
Gratification, Motivation for Vocational Achievement j Attitude Toward 
Authority, and Job Holding Skills. All of these deal with thi individual' 
perceptions of himself and what he believes would constitute appropriate' 
behaviors in various situations* Here the sex difference of note Is a 
tendency for the factor (among males only) to pick up a slliht cognitive 
component with interpre table levels of loadings on Job Knowledge (,39) 
and Job Seeking Skills (.30) t Not only has a similar sfiK difference in 
the loading patterns been found for this particular factor, In results 
with N.Y.C, samples, but the same three factors discussed above have also 
appeared as the dominant ones in those previous analyses (Preeberg, 1968, 
1970; Freebarg ^ Reilly, 1972). 

Pre- and Post-Score Changes 

Mean scores at OIC ent^ (Pre-Feeder course) and following the four 
to six week Feeder training are shown in Table 4. Very little change was 
observed in the test scores, although they tended to be in the expected 
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Table 4 

Test- Re test, Pre- and Post-Feeder Program 
Means and Standard Deviations 

(N « 107) 







Pre- 


-Teat 


Post- 


■Test 




Test 


M 


0 


M 


g 


1. 


Job Knowledge 


19.7 


2.7 


19.8 


2,6 


2 . 


Self-Es teem 


38.5 


5.2 


38.9 


4.1 


3. 


Deferred Gratification 


52.9 


6.7 


53.0 


7.4 


4. 


Level of Voc. Asplr. 


208.4 


28.0 


205.2 


28.0 


5, 


Level of Voc. Plans 


167.1 


36.1 


176.9 


37.0* 


6. 


Aspirations minus Plans 


40.8 


33.5 


28.9 


37.8 


7. 


Attlt. toward Authority 


44.5 


8.9 


45.9 


8.6 


8. 


Job Seeking Skills 


12.4 


2.9 


13,0 


2.7 


9. 


Job Holding Skills 


30.6 


2.8 


31.1 


1.8 


10. 


Motlv. Voc, Achlevenient 


53.7 


14.2 


60.5 


13.5^ 


11, 


PR-Map Eeadlng 


5,4 


2.5 


5.9 


2,8 


12, 


PR-aip Coding 


7.7 


2.2 


8.2 


1.9 


13. 


FR-Flle Card 


9,1 


1.3 


9.6 


0.7 


Interest: 










14. 


Aesthetic 


11.3 


2,3 


11.4 


2.4 


15. 


Science 


11,2 


2.6 


11.4 


2.4 


16. 


Technical 


10.0 


,3.0 


10 .3 


2.7 


17. 


Clerical 


12.7 


2,2 


12.3 


2.4 


18. 


Business 


11.9 


2.4 


12,0 


2.3 


19. 


Service 


12,7 


1.9 


12,1 


2.1 


20. 


Outdoor 


7.8 


2,9 


7.9 


2,7 



* significant at ,01 level 
^ Significant at ,02 level 
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direction as indicated by the higher mean score after the Feeder Program* 
However, only two of the mean changes represent significant improvemenc. 
These appear for the higher Post- Feeder Motivation for Vocational Achieve- 
ment score (CR ^ 4.7; p. < .01) and Level of Vocational Plans (CR * 2,52; 
P - .0.2). 

These results are presented only for cursory examination sit^^e^ as 
indicated previously, the relatively email smple of 107 may be biased In 
a number of ways. In addition, little change in the constructs measured 
could be expected to occur over such a short time span* Under any clr^ 
cumstance, the contrasts from one ti™ period to mother remain difficult 
to interpret without some form of control group for comparative purposes # 

B . Criterion Dimensions 

The sets of 30 Program Completion and 35 Poat-Program criteria were 
each factor analysed using a principal components solution with Varimax 
rotation to orthogonality- In the case of the Program Completion criteria , 
the sample si^e of 304 allowed for separate factor analyses by sex. The 
Post^Program criterion sample of former trainees who had obtained full- 
time employment (and for whom the full range of job dependent criterion 
variables were applicable) , however, dictated the necessity of combining 
males and females into a single sample of 103 » 

Program Completion (Short-Term) Criterion Dimensions 

Interpretable criterlori factors and their loadings, as extracted 
from the 30 x 30 intercorrelation matrices for males and females, are 
shown in Tables 5 and 6, Along with each set of loadings, the total 
factor variance accounted for is shoTO as an indication of factor 

-23-- 
Q A 



Table 5 



Rotated Loadings for 
Program Completion Crltarion Factors 

(Males; N * 115) 



Factor I 
Training Progratn Adjustment 



Variable Loading 

Voc, Skill Instruccor 

Rating ^74 

// Absences — *65 
Remedial Instructor 

Rating ,50 

Troubla with Police -.40 

Peer Rating ,37 

Feelings of Voc, Adequacy ,35 

Counaelor Rating ,29 

Factor Variance ^2,2 



Factor II 
Person al-Social Adjustment 



Variable Loading 

Cat Along with Family .76 
Awareness of Job 

Characteristics p72 

Save Money ,55 
Family Feelings About 

Trainee , 45 

Peer Rating ^41 
Importance of Keeping Out 

of Trouble ,31 

// Paople Giving Hard Time -,30 

// Health Prpblems . -.3o 



Factor Variance ^2,4 



Factor III 
HDnetary Expectation 



Variable 


Loading 


Starting Salary Expected 


.89 


Top Salary Es^ected 


.86 


Least Pay for Desired Job 


.86 


Ramedial Instructor Rating 


.55 


Ways to Look for Desired 




Job 


-,33 


Financial Help to Family 


• 31 


Factor Variance ^3,2 





Factor IV 
Effective Job Planning 

Variable LQadln 



Level of Long-Term Job 



Plans 


,67 


Wiat to do beyond First 




Job 


.63 


Important Things to Ask 




Job Interviewer 


,59 


// Relevant Ways to Search 




for Job 


,55 


// Relevant ReaBons for Sal, 




Present Training Area 


,54 


Feelings of Voc, Adequacy 


*44 


Coimselor Rating 


,30 



Factor Variance ^2,5 
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Tab la 6 



• 


Kotated Loadings for 




PrograTTi 


Lompietion 


Criterion Pactora 






(Feinalea | 


N « 189) 




Factor 1 




Factor II 




Training Prpgram & Social A< 


ilustmant 


PogitlvB Job Orientation 


Variable 


Loading 


Variable 


Loading 


Family Feelings About 








Trainee 


.64 


J QD 


ft! 


Counsalor Hating 


.64 


Long Range Job Plana 


.66 


Get Along with Family 


.61 


Awareness of Job 




Peer Rating 


.56 


Characterigtlcs 


.45 


Remedial Instructor 




AHi 1 i f-v ffi Car^T'^ Hiif Pi an«^ 


43 


Rating 


.53 


Re lev AH to Search 




# People Giving Hard Time 


-.53 


for Job 


.41 


Feelings of VoCi Adequacy 


.47 






Voc* Skill Instructor 




Tf^?^ Tt^ %»^f ^ fif^TS 

J Up j-ilt^rviewei 




Rating 












ractor variance - ^ • j. 




Factor Variance ^ 2*6 








Factor III 




Factor IV 




Monetary Expectation 




Vocational Confidence 




Variable 


Loading 


Variable 


Loading 


Amount of Pay Acceptable 


.85 


Job Knowledge 


.68 


Starting Salary Expected 


.84 


Ability to Perform Job 


.65 


Top Salary EKpected 


.83 


Save Honey 


.59 


Level of OIC Training 


.32 


# Relevant Ways to Search 




# Jobs Willing to Train 




for Job 


-.38 


For 


-.32 • 


Feelings of Voc, Adequacy 


.37 


Voct Instructor Rating 


-.32 


Job MoClvatlon 


.33 






Level of OIC Training 


-.30 



Factor Variance » 2.8 



Factor Variance " 2,1 



ERIC 
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dominance. As before in the analyses of the test battery, loadings of 
approximately .30 or greater are utiUzad for factor interpretation. 
Although factor designations for tnaleB and females reflect broad slinl- 
laritles, thera are a number of dlfferancei In the specific loading 
patterns. Factor I in each aample ( Training Program Adjustnent for 
males and Training Proa ram and Social Adiustmenc for fenalas) is clearly 
one that defines trainee behaviors indicative of succasfl in the program 
based on ratings by peers , counselors, remedial and vocational skill 
instructors and (for males) absences over the course of training. Only 
one aapact of social adjustment accompanies program success for males in 
the form of "Trouble with Police"; whereas, females ihow a much broader 
component of family and connnunlty adjuatnents along with their adjustment 
to the training program (thus, dictating the somewhat different designa- 
tion for the female v rslon of the factor) . Social adjustment variables 
for males combine to form a readily Interpretable and separate factor of 
Personal-S oeial Adiustment (Factor II) that defines a trainee who Is ad- 
justing in terms of family, connnmity and peers i In his perceptions of 
work, his need to avoid trouble and his asseaament of his personal health. 

The factor that is dominant in each sfflcnple, in terms of factor 
variance ( Monetary jxpectatlona ; Factor III) is quite similar for the 
sexes; being definable almost exclusively from the high positive loadings 
on salary expectations (starting salary, top salary and nilnimum pay expec- 
tations for a desired job). Among males, those who have their sights set 
on higher earnings tend to receive hl^er ratings by one of the Instruc- 
tors (Remedial Skills Instructor). Females with such ambitions, by con- 
trast, show a slight tendency to receive a lower rating (i.e., from the 



ERIC 



-26- 



37 



Vocational Skills rnstruccor) / Other negative aspects of perfoimance 
also enter into the factor, to a minor degree, for both sexes , For mnleSj 
those scoring high on the factor show lesser knowledge of effective ways 
to search for a desired job* For females, there la an unwillingness to 
volunteer to train for a variety of different jobs ^Ith high monetary 
expectations . 

The retnaining criterion factor for males has a vocational focug 
that depicts Effective Job Planning (Factor IV) dominated by knowledge 
of what to do if the first job obtained after OIC doesn*t work out, a 
highar level of long-term job plans i knowing important questions to ask 
a job Interviewer about a prospective job| the best ways to search for 
a job and more relevant reasons for having selected the particular voca-- 
tlonal area for training In OIC* This pattern beari a marked resemblance 
to Factor II for females, which carries the same deilgnation and dominant 
loadings on most of the same variables* 

Relatively unique to the female sample in its ioadlng pattern is 
Factor IV, Vocational Confidence , which defines a trainee who is confident 
of her knowledge of a desired job and her ability to perform itj along 
with a positive attitude toward work in the form of stronger Feelings of 
Vocational Adequacy and Job Motivation* However * th«fe confidence may be 
somewhat misplaced (i*e» , overconf idence?) since the trainee appears to 
be less capable of describing relevant ways to search for Jobs* Moreover, 
these women tend to be enrolled in lower status level occupational 
courses In OIC* 

* Areas of training were rated for status level on the basis of the NORC 
Occupational Status Scale (Duncan, Hatt a North, 1961). 



Although some few criterion variables used here differ from those 
used with previous (N,Y,C,) study samples, the resulting criterion factor 
designations dealing with training program adjustTnents , social adjustment, 
and planning capability were fOLmd to be equally present and Interpre table 
for those younger school dropout trainees of the N.Y.C. prograni-=with 
specific loading patterns being only approximately comparable* The 
difference of primary Interest is that the Monetary EKpectation factor 
which appears as dominant for OIC males and females was not at all evident 
in the N.Y,C. sainples. 

Post-Program (Longer- Term) Criterion DiTnensions 

From the 35 x 35 intercorrelatlon matrix of post'-program criterion 
variables (both sexes combined) four interpretable factors were extracted 
and are presented below (Table 7) in terms of their designations and 
loadings * 

The most clearly interpretable and dominant factor is the one 
designated as Overall Employment Success (Factor I) with its primary 
loadings on the variables of Salary Raise , Promotion, Job Satisfaction, 
Length of Stay on the Job, Saving of Money, and Current Employmant 
Status (i.e., at the time of the interview). 

Another md somewhat different form of job success appears as 
largely specific to the worker who entered a blue collar employment 
setting ( Blue Collar Job Success ; Factor II), This cluster of criterion 
variables depicts a former trainee who entered a blue collar Job setting, 
obtained a higher level Job, higher starting salary, expressed greater 
Job satisfaction, had higher Job pay expectations for the future ^ and, 
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Table 7 



Rotated Loadings for Post-Program Criterion Factors 

(N ^ 103) 



Factor 1 




Factor II 




Overall Employment Success 


Blue-Collar Job Succas 


is 


Variable 


Loading 


Variable 


Loading 


Save Money 


.59 


Top Pay Expected 




Raise in Salary (No/Yes) 


,58 


Job Quality Laval 




Met Job Expactations 


.55 


Level of Future Job 




Job Satisfaction 


• 55 


Sought 


.63 


Employed (At Time of 




AraoOTit of Starting Salary 


,58 


Interview) 


.54 


Job Satisfaction Scale 


.49 


Promotion (No/Yes) 


.54 


Blue/White Collar Employ- 




Length of Stay on Job 


.47 


ment 




Family Feelings about 




Met Job Expectation 


,40 


Trainaa 


.43 


# Contacts to Find 




Ways of Finding Next Job 


.36 


Trainee 


-.35 


Factor Variance ^ 3 


1,1 


Factor Variance ^ 3,0 





Factor III 
Job-'Seeking Effort 




Factor IV 
Social Adlustment ^ Job 


Planning 


Variable 


Loading 


Variable 


Loading 


// Peaces Interviewed 


,82 


Long Range Job Plans 


• 72 


# Applications Filed 


,80 


# People Giving Trainee 




// Sources used to Find 




Hard Time 


-•55 


First Job 


,54 


Ways to Carry Out Job 




Received Unemployment Pay^ 




Plans 


,48 


ments (No/Yes) 


,38 


Financial Help to Family 


.47 


Length of Stay on Job 


-.36 


Going to School (No/Yea) 


.45 


Knowledge of Desired Job 


-.36 


Blua/W^ita Collar Employ- 




Hours Worked/Week 


-.35 


ment 


.40 






Employer Rating 


.35 


Factor Varlmce - 2,7 




Hours Worked/Week 


-.32 



Factor Variance ^2,3 
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If he left the present Job, would tend to seek one of higher status 
level. Factor III. Interpreted as one of Job Seekln. Effor. ,eems 
reasonably definable on the basis of its highest loadings on number 
of places interviewed, flppllcations filed, and aourcei used to find 
a job. The diligent search, however, does not characterise someone 
for who. the effort has necessarily paid off. in that this former trainee 
is one who has worked fewer hours per week, spent less time on the Job 
he held (or holds), and has tended to receive more unemployment insurance 
payments • 

The fourth factor (Social Adiustment and Job Pl^nnin.^ u not as 
clearly interpretable as the first three, since it seems to incorporate 
elements that would be thought to constitute somewhat separate dimensions.* 
The Job planning aspect of the factor is apparent in its laadlngs on long 
range plans, knowing how to carry out Job plans, and attendance at school 
on the part of the former trainee who tends also to be the one employed 
in a white collar setting. Coupled with these are the obvious social 
adjustment indices of providing financial assistance to the family and 
better community adjustment (i.e., fewer people giving the eK=tralnee a 
"hard time"). Those who score high on this dimension receive a higher 
employer rating. Paradoxically they are somewhat more likely to work 
fewer hours per week, 

C. Test Validation 

Having identified the criterion factors, the predictive test validi- 
ties-which are of major Interest for present study purpoae8-=can now be 

* t^^ti °^ the variables that load on the factor are probably sex= 
Se^? (^-8;. "Blu« Collar /White Collar Employment" correlates !32 with 
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determined using criterion factor scores eomputed from those short'- term 
(program completion) and longer-tenn (post-program) dlmenalona. As 
Implied previously, the greater burden of evidence for demonstrating 
validity must reside primarily with the program completion criteria, 
because of more acceptable sample characteristics, better analytical 
possibilities that stem from the larger sample slie, and the opportunity 
to conduct separate analyses by sex, 

Short-Term (Program Completion) Validities 

Validity coefficients for the 13 test scores and the Vocational 
Interest aubscales, using factor scores obtained from the Program Com- 
pletion matrices, are presented in Table 8 for males and Table 9 for 
females . 

Significant validity at low to modest levels for a number of the 
measures is evident— especially in relation to the Training Program 
Adjustment dimension. For males the highest predictive validity with 
that criterion factor is found for the attltudlnal measure that reflects 
awareness of how to behave in the job setting (Job Holding Skills; 
r - .34)1 while two cognitive measures also achieve respectable levels 
of validity (Job Knowledge; r ^ ,28, and PR-File Card Sortingi r ^ ,24) • 
Self-Esteem (r - ,20) and Deferred Gratification (r ^ .22) add additional 
evidence pointing to the minimal but significant value of other attltudl* 
nal perceptions* 

Overall, higher levels of test validity are foimd for the female 
sample, among whom the Job Holding Skills test (r = .36) qualifies as 
the most valid measure of the battery, just as It did for males. Job 
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Table 8 

Validities for Test Battery Using Progrra Completion Dinehilona as Criteria 

Males (N » 115) 







Factor 1 


Factor 11 


Factor 111 


Factor 






Training 


Personal-- 




Effect" 






Program 


Social 


Monetary 


Job 




Test 


Adjusttnent 


Adjus tment 


Expectation 


Flannel 


1. 


Job Knowledge 


,28** 


.19* 


.01 




2. 


Self-Esteem 


.20* 


• 13 


- 03 


1 h 


3. 


Deferred Gratification 


,22* 


,09 


- 03 




4. 


Level of Voc, Aspir, 


p02 


-,03 


. 11 




5, 


Level of Voc* Plans 


-.05 


.02 






6,^ 


Aspirations minus Flans 


*08 


-.05 


- 07 




7. 


At tit, toward Authority 


• 07 


.18* 


.15 




8, 


Job Seeking Skills 


• 15 


*20* 


,12 


.20* 


9. 


Job Holding Skills 


,34** 


.19* 


.03 


.06 


10, 


Motiv, Voc, Achievement 


fi7 


*07 


.05 


.06 


11, 


PR-Map Reading 


.13 


• 00 


.01 ' 


.05 


12, 


PR-Zip Coding 


,17 


.06 


.05 


-.05 


13, 


PR-File Card 


,24** 


-,06 


-.12 


-.02 


Interest I 










14, 


Aesthetic 




- ,IL 


-.04 


-.12 


15. 


Science 


.07 


.11 


.08 


.14 


16. 


Technical 


.00 


-.01 


,24* 


.01 


17. 


Clerical 


-,02 


.14 


-.06 


-.09 


18. 


Bus Iness 


-.07 


.02 


-.02 


-.04 


19, 


Service 


,07 


.16 


.01 


.04 


20. 


Outdoor 


-,02 


-.02 


.16 


-.04 



* Significant at .05 level 
** Significant at ,01 level 
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Table 9 





vaJ,i,ai.cies tor iesc oaccery 


Using Program Completion 


Dimensions aa 


Criteria 






KeinaleH (N 


■ 189) 










Factor I 


Factor II 


Factor III 


Factor IV 






Training 












Program 


Positive 




Mon ft t arv 






& Social 


Job 


Vocational 




Test 


Adjustment 


Orientation 


Confidence 


Iscpectatlon 


1. 


Job Knowledge 


.25** 


.28** 


.03 


.14 


2. 


Self-Esteem 


.16* 


.01 


.09 


.05 


3. 


Deferred Gratification 


.16* 


.17* 


.12 


.05 


4. 


Level of Voc. Aspir, 


-.13 


-.18* 


-.19* 


-.05 


5. 


Level of Voc. Plans 


. -.36** 


-.18* 


.07 


-.06 


6. 


Aspirations minus Plans 


.25** 


.02 


-.22* 


.02 


7. 


Attit. Coward Authority 


.14 


.04 


,03 


-.02 


8. 


Job Seeking Skills 


.24** 


.37** 


.15 


,10 


9. 


Job Holding Skills 


,36** 


.21* 


-.04 


.25** 


10. 


Motiv. Voc. Achievement 


.03 


-.03 


.02 


-,03 


11. 


PR-Map Reading 


.16* 


.21* 


-.01 


.05 


12. 


PR-Zip Coding 


.23* 


.14 


.03 


,17* 


Interest: 










13. 


Aesthetic 


-.12 


-.16* 


-.11 


-,11 


14. 


Science 


.09 


-.02 


.04 


-.02 


15. 


Technical 


-.17* 


-.20* 


-.06 


.06 


16. 


Clerical 


-.10 


-.03 


.05 


.09 


17. 


Business 


.15 


-.14 


.04 


.02 


18. 


Service 


-.03 


-.07 


-.15 


-.01 


19. 


Outdoor 


.03 


-.01 


.01 


.05 



* significant at ,05 level 
** Significant at .01 level 
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Knowledge (r - .25), Job Seeking Skills (r « .25), and PR-Zip Coding 
(r = .23) rapresent the cognitive contribution to validity in their pre- 
dictive value against the "PrGgram Adjustment" cluster of criterion 
behaviors. Of interest in the test validities with this factor is the 
coefficient for Level of Vocational Plans in the form of a relatively 
substantial negative r of -.36. Along with this Is the significant 
positive value (r - .25) for the Asplratlons-Plans discrepancy score 
(LVA minus LVP). OIC females with high levels of vocational expectation 
(plans) , are the ones least likely to display the behavior pattern that 
defines success at program completion. This apparent disadvancageous 
effect of too high a level in occupational goal expectations for females 
is reinforced by the two negative validities for LVA and LVP with the 
"Positive Job Orientation" factor (r's - -.18). At the same time, those 
trainees whose job hopes (aspirations) exceed their expectations (plans) 
to a greater degree=-i.e. , where vocational plans are kept well below 
vocational desires=-tend to be the ones who show a higher level of train- 
ing program success. 

With the "Positive Job Orientation" factor, both Job Knowledge 
(r = .28) and Job Seeking Skills (r - .37) measures are again found to 
be dominant in their validities, as they were for the Training Program 
and Social Adjustinent factor. Other scattered significant validities 
for males and females on the remaining criterion factors are self-evident 
and relatively minor In magnitude. 

In terms of the overall valldlcy patterns, the most effective of the 
tests for males would appear to be Job Knowledge, with its predictive 
capability across three of the four criterion dimensions, followed closely 



by Job Holding Skills and some lesser degree of value for the Job Seeking 
Skills measure. For females, the beit of the measures on the battery are 
essentially, the same three as for the niales but with considerably higher 
validities and somewhat different rmking"i.e.. Job Holding Skills is 
clearly of first rmk^ followed closely by the Job Seeking Skills and Job 
Knowledge measures. The tests of Practical Reasoning show only minor and 
scattered indications of validity for these short-term outcomes, with 
somewhat more promise for females than males # 

The Interest subscales have the least demonstrable value with these 
criterion dimensions | nor would they necessarily have been expected to 
show strong validity patterns, without the opportunity to examine the 
relationships within specific VDcational ekill areas in the OIG program 
(an impractical approach given the presently available data) ** 

Longer-Term (Post-Program) Validities 

Validities for the test battery, based on Post^'Program criteria p 
are shown in Table 10. The levels of validity achieved are not particu- 
larly striking for this relatively small md seriously restricted follow- 
up sample of 103, that required cortsinlng of males and females for any 
useful analyses. Despite the probable sair^le limitations, however, the 
results show certain patterns of relationship that merit coottsent* One 
such pattern tends to confirm the earlier indication of a negative rela-^ 
tlonship between occupational plans and e^loyment success (r's ^ -*22 
and -.23 with the two employment success criterion factors) and is coupled 

* Lokan (1973) reports greater predictive validity than shown here for 
these Interest scales when grades in particular vocational skill areas 
of secondary school programs are used as the criterion. 



Validities for Test Battery 



Table 10 

Using Post-Pronrara Dimensions as Criteria 
(N * 103) 



Interest.- 

14. Aesthetic 

15. Science 

16. Technical 

17. Clerical 

18. Business 

19. Service 

20. Outdoor 

* Significant at .05 level 



-.21* 

-.OS 
■,20* 
,03 
,01 
.17 
,03 



-.03 
.04 
.08 
,00 
.17 
.23* 
.16 



-.04 
-.07 
',09 

-.14 
.01 
.03 
,14 
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Factor 1 


Wan^rs'^ TT 

ractor LL 


Factor III 


Factor IV 
Social 
Adjustment 
& Planning 




Test 


Overall 
Employment 
Success 


Blue 
Collar Job 
Success 


Job 
Search 
Effort 


1. 


Job Knowledge 


,18 


-.07 


.07 


.13 


2, 


Self-Esteeni 


--.02 


.09 


. ,04 


.04 


3, 


Deferred Gratification 


.06 


.17 


-,02 


-.02 


4. 


Level of Voc. Aspirations 


-.03 


-.10 


-.06 


-.08 


5, 


Level of VoCp Plans 


-,22* 


-.23* 


-.00 


,00 


6. 


Aspirations minus Plans 


,21* 


,17 


-.06 


-.06 


7* 


Attitude toward Authority 


.11 


.28* 


-,05 


.10 


8, 


Job Seeking Skills 


.11 


.20* 


.05 


,17 


9. 


Job Holding Skills 


.05 


.00 


-.07 


.00 


10, 


Motivation Voc. Achievement 


,05 


.17 


.08 


-.18 


11. 


PR-Map Reading 


,11 


.14 


.19* 


.20* 


12, 


PR-Zip Coding 


.18 


.29* 


,08 


,15 


13, 


PR-File Card 


,05 


.13 


.00 


-.08 



-,17 
-.03 
-,02 
-.04 
-.03 
-.04 

.00 



with an Indication that favorable outcome attaches to the trainee who 
mnintninH hlH nrcupfltlonal defllres (agplratlon.^i) well bayond hlM nctufi] 
job plans** 

Two of the cognitive skill tests (Job Seeking Skills, r ■ .20; and 
PR-Zip Codings r ^ *29) provide modest levels of validity with the most 
predictable of the post-prograffl criterion factors^ i,e*| the "Blue Collar 
Job Success" factor. Attitude Toward Authority, In its relation to that 
same criterion factor (r ^ .28) , represents the one attitude scale with 
any worthwhile level of validity * For the two remaining criterion perfor- 
mance dimensions of "Job Search Effort" and "Social Adjustment," PR-Map 
Reading yields the only two significant validities foimd. 

Scattered and more difficult to Interpret validities seen in Table 
10 for the Interest subscales could be considered an imderstandable 
result of having to combine male and fefnale sables on measures known 
to be sensitive to sex distinctions* 

Emp loymen t as Cr 1 te r Ion 

An opportunity to exatnine "Employment" as a post-'program criterion 
is possible with the availability of the 103 members of the sample who 
obtained full'-time employment (one weak or more) following their OIC 
training and the 54 who had not found employment during the 6 to 8 month 
follow--up period. Bi-serial correlations between thi dlchotomous employ- 
ment variable (Employed vs. Not Employed) and each of the 20 predictor 
test scores are shown in Table 11. 

* In prior study results, using N.Y.C. trainees (Freeberg, 1974) ^ a simi- 
lar level of negative predictive validity was found between the LVP 
measure and "Blue Collar Job Success" (r ^ -,21) as well as with one 
other job performance criterion factor (r ^ -.31 with a factor of 
"Short'-Range Job Orientation") , 
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Table 11 

Predictive Validities Using Full-^Time Employment 
(Employed Full Tlttie vs. Never Kmployed) 
as Criterion 

(N ^ 157) 

Test ^bis 

If Job iOiwledge -**08 

2. Self-Esteem ,14 

3. Deferred Gratification .13 

4. Level of Voc. Aspirations -.04 

5. Level of Voc. Plans ,05 

6. Aspirations minus Plans -.04 

7. Attitude toward Authority ,20* 

8. Job Seeking Skills ,21** 

9. Job Holding Skills .06 

10, Motivation Voc. Achievement -.09 

11, PR-Map Reading ,15* 

12, PR»zip Coding ,12 
13- PR-File Card^ ,33* 
Interest I 

14. Aesthetic .^5 

15. Science 

16. Technical ^iq 

17. Clerical ^07 

18. Businass 

19. Service 

20. Outdoor ^^.l 

^ Males only> K ^ 43 

* Slgniflcmt at the .05 level 

* Significant at the ,01 level 
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The two testa haying the most sl^lf leant levels of validity with 
this employment criterion are Attitude Toward Authority and Job Seeking 
Skills. This tends to strengthen their value at long-terra predictors » 
previously shown above, with the "Blue Collar Job Suceeis" dimension as 
criterion. It seems logical that thoie who posseis Qapabillty in knowing 
where and how to seek employment would be the ones most likely to find 
it, while a inore positive (l,e., "acceptable") attitude toward those In 
positions of authority also appears to add to that likelihood, Some 
prospects for the value of cognitive akill capability as an employment 
predictor also emerge in the r of *33 for the PR-File Card Sorting test 
(males only) and is reinforced slightly by the minimal but significant 
r of *16 for PR-Map Reading. 

CONCLUSIONS 

In this attempt to validate a test battery for uae with trainees 
in a manpower program that stresses vocational skills development (l.e.p 
OIC), it has been possible to demonstrate the extent to which a number 
of the measures show promise as valid assessment tools. As might have 
been anticipated p however, the validity aehleved tends to be largely a 
function of the nature of the criterion measures chosen, their temporal 
position (as short-tarm or longer-term outcomes) ^ the quality of the 
criterion samples or data obtainable, and the sex of the respondents 
for whom validity is ascertained* 

Such considerations argue for limiting the conclusions of the 
present study primarily to the findings based on the short-term (program 
completion) criteria. Deficiencies In the post-program criterion data 
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sevarly limit the generalliatlona that can be drawn from Cheae findings. 
Given the Foregoing eaventj It can be Qoncluded thati 

(a) Predlctiva validity of the battery wai best against empirically 
defined, independent clusters of short-tem perforiBance out- 
comas (l,e.j criterion dlmenslong) obtained at training program 
completion. 

(b) Fairly consistent significant validities, at modest levels 
(r's ^ ,20's to mid iSO's), appeared for those cognitive testa 
dealing with knowledge of the requirements entailed In perform^ 
Ing a variety of jobs (Job toowledge) , knwlng to search 
for and apply for jobs (Job Seeking Skills) and in following 
directions for job-slmulated tasks (Practical Reasoning Skill)* 

(c) In the attitudlnal domain i superior validities belong, unmis- 
takably, to the measure that reflects an awareness of appro- 
priate behaviors in a job setting (Job Holding SkUls)" 
essentially on a par with the measures of cognitive ability, 
Self-^Esteem, Deferred Gratification and Attitude Toward 
Authority scales reach validity levels that are significant 
but minimal In magnitude and, as a group, fall disclnctly 
below validity achieved with cognitive measures* 

(d) Validities obtained for a measure of trainee vocational plans 
indicate that females whose occupational expectations are 
higher tend to be less successful In their criterion perfor- 
mance at the completion of the program* For both sexes 
combined In a single sample a similar result, with criteria 
of pose-program employment and job success, was obtained. 
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(e) Interest iubscales showed little predictive validity against 
the broad criterion dlmanslons, Slnee thsai dlmenBlons did 
rfot reflect specific jobs or different occupational areas » 
a more definitive assessment of validity would seem to be 
' . contingent upon data that allowed for determining differential 
validities of the various Intarast scales against subgroups 
of occupations entered* 
Cf) Those short-^term criteria of success for which the test 

battery is most predictable are the ones thut fonn a dimension 
of "Training Program Adjustment" composed of ratings of 
counselors, instructors, and peers ^ along with several measures 
of adjustment to family md comnmilty. Thli would appear to 
be tha dimension for which teat prediction of short-- term per-- 
formanca is logically rmmt desirable and tot which the data 
are rather readily obtainable, 
(g) Using factorially derived post-program criterion dimensions^ 
designated as "Overall Employment Success," "Blue Collar Job 
Success, "Job Search Effort," and "Social Adjustment and 
Planning" only a scattered handful of significant predictive 
test validities could be fotmd* These were primarily in 
relation to the "Blue Collar Job Success" crttarlon dimension 
for the tests of Job Seeking Skills, Attltud'fe Toward Authority^ 
and a measura of Practical Reasoning, As fuirthar confirmation 
of the value of these measures they are also the ones that have 
positive predictive ralatlonshlps with the single dichotomous 
criterion measure of whether or not the trailtee obtained full-- 
time employment after leaving 01 C, 
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(h) Measures of the battery may be considered generaliy applicable 
to adult trainees In programs that stress Dccupational skills, 
as well as to the younger, adolescent * iroups found predomi- 
nmtly in "work-experience" manpower prograTOS (for whom the 
tests were originally developed) • Based on scale flcores, 
dimensional structure of the battery and overall validity 
patterns^ including the criterion dimensions derivad for use 
in validation, a reasonable level of similarity could be said 
to eKist in the test performance of these groups. The similar- 
ities occur despite numerous differences in sample compoaition 
with regard to education^ age, marital status, and the eco= 
norolc conditions during the time periods in which data ware 
ob tained . 

From these conclusions, the Infersnce may be drawn that the mast 
valid of the tests are likely to be useful as guidance tools for opera= 
tlonal assessment purposes In manpower training programs* On the basis 
of findings for both concurrent validity studies (Freebergi 1968; 
Freebarg a Rellly, 1972) and short-term predictive validity studies 
(Freeberg, 1974), the dominant measures most difenslble for programmatic 
use have continued to be the cognitive ones of Job toowledge. Job Seeking 
Skills, and Practical Reasoning abilities, along with the attitudlnal 
scales of Job Holding Skills ^ Self-Esteem and Attitude Toward Authority, 
Other tests that define logically relevant constructs remain tenuous until 
further validation atten^ts can be made with better quality criterion data* 

In addition to the use of these tools for guidance or placement pur- 
poses (e*g,, to Identify "high risk" Individuals in a formal counseling 



framework) , the best of the measures might also be used in an heuristic 
way for defining areas of strength or wealmesa in trainee groups (e,g*, 
at program entry) and thus aid in modifying eurrlculum content. Addition- 
ally^ in situations where there la substantial eongruence between program 
obiectives and constructs Masured by the tests, there Is some justifica- 
tion for their use as TOasures of change result Ing from exposure to the 
training program (given proper experimental control in any 'Tsefore-^wid- 
after" study design). Finally, the short-tara md longer-term criterion 
dimensions that define Independent ^ multi-faceted aspects of trainee 
parforniance (derived here for test validation purposes) can also provide 
useful coherent scales for assessing program effectiveness over time. 

Any operational use of the most valid of the tests, even on an 
experimental basis, would. In any event, require compilation of some 
form of user's manual that provides separate sets of normative informal 
tlon for an adolescent population in a work-axperlenca program and for 
an older population of trainees in an occupational skills type of pro- 
gram. In conjunction with those data* there should be guides to broad 
interpretation of each measure (or score profiles for sets of measures) 
indicating ranges of likely trainee criterion performance based on the 
level of test validities so far demonstrated. 



54 

-43- 



_aEIEEEHCES 



Backer, T, E. Mathodi of aisessitig the dlsadvantaggd lii manpower programs. 
A revtew and analyals. Los ^geles , California^ Human Interaction 
Research Institute^ Noven^er 1972, 

Barnes, H, Finding and interviei^lng the hard-to-locate ^ Hie DMI 

ascperience. In E, Boms (Ed,)* Evaluating the Impact of manpow er 
pr&graina , Lexington, Mass,: C. HeathTTfT?; 

Borus, H, E, ^ Taah , W. R, Measiiring the impact of manpower p ro eranis ; a 

Ann Arbor, Mich, t Institute of Labor and Indus trial Ralatloiis , 



The University of Mlchlgan-yayne State University, 1970, 

Duncan, 0., Hatt, P,, ^ North, C. gccupatlons a nd a ocial status , Naw York' 
Free Press of Glencoe, 1961, . 

Econoinlc Op portunity Act of 1964 , as amended. Washington, D, C.i Office 
of Economtc Opportunity, Decainbir 1966 • 

Federal Raglste r Title 29 - Ubor , Part SO^-Neighborhood Youth Corps 
Projects, Washington, D, C, ^ December 1969, 34 (No, 230)* 

Freebarg, N. E* DavelDpmant of evaluation measures for use with neighbor^ 

hood youth corps anrollees ,_ . EroJact Report 68-^ Princ€t0nr N * J . r 

Educational Testing Service^ 1968. 

Freaberg, E, Aseassuient of disadvantaged adoleictnts i A diffarent 

approach to research and evaluation measuras* Journ al of Educational 
Psychology , 1970, 61, 229-240, ' 

Freaberg, N. E. Development of assessment meaaurefi for use with youth-work 
training prograsi enrollees , Phase II: Lonittudlnal Validation, 
Project Raport 74-1. Princeton, N, J, i Educational Testing Sarvlce 
1974. ^ ' 

Fraabarg, E. ^ Rellly, R. Validation of a teat battery for youth-work 
training program enrolleas, Reaaarch Mamorandum 72-'ll. Prliiceton, 
N. J.: Educational Testing Service, 1972* 

Kaiser , H. F, The varlma^c criterion for analytic rotation In factor 
analysis* Paychomatrlka , 1958, |3> 187-200* 

Karpt J. M. & Slgel , I, Psychoeducatlonal appraisal of disadvantaged 
children, Ravlaw of Educational Researc h^ 1965 ,^^ 401-412. 

LennDns 1. Testing and the culturally dlgadyantaged child . New York ^ 
Harcourtj Brace a Worlds 1964, " " " 



-44- 



55 



Levltatij S- A, I & Ming\roi 0. L. Fgdetal tralnlii^ and ^ork progr axne 1ti 
the sixties^ Part V, Mti Arbor, Mich, V institute of Labor and 
Induscrial ReLaticin.Sp 1969* 

Lokan , J. Ji Thm development md prello±tta.ry evaLuatlon of an InstTuiiiiiit 
to measuTfi the vocational interests of low-Teadlng high-echool 
students. Paper presented at 1973 Azinuml Coafareiice of NM, New 
Orleans^ February 1973* 

National League of Cities. Focus oa nianpower plaiinlngi 2-Monl toeing and 
evaluatlQii , VJashlngton, D, C: Office of Urban Servlcea ^ Decfimba^ 
19 73. 

Opportuiitttas Industrialization Canters of iVmerica . Annual Report 1975* 
PhlladeLphla, Ja, ^ 19 75* 

Society for the Psychological Study of Social Xsaues, Guidelines for 
testing minority group children, journal of Social Iseues, 1964. 
12 7-145, ^ 

United States Departmerit of Labor, A _ compilation of papers on data- 

gathering In manpower res earch. Unpublished sianuscrlptj September 
1969, (Available from Office of Manpower Research^ yashingtotij 
D. CO 

United States Dapartmeat of Labor, Standards—yeighborhood Youth Corpi 
Out-Qf-.S Qhpo I program , N , Y , C ,-2 / "Wtshlngtoft, D, C* , April" 1971. 

Unitad States Office of Eeoncniic Opportunity^, Office of Planning , Seaaarch 
and Evaluation* Federal Youth Pro gra ms ^ A discussion papers Da camber 
1972 , 

iniol^y , J, S,^ et al, Fader al evaIuatl_oti jollcy > Washington^ D,C,: The 
Urban Institutes 1970, 



-45- 



56 



JBNDIX A 

Program Ccmpletloti CrlteTlon Vartablea 



(1) Faellnga of Vocational Adeqtiacy (7 Item scaLg sCQre) 

(2) Status Level of Occupatiom Chosen for rrainLiig 

(3) Ralavance of Reasori for Selection of Occupational TTaining Choice 
( I r re le ^an t / Re le van t ) 

(4) Awareness of Job Characteristics (8 Itein acale score) 

(5) Kno\^ledga of Desired Job afcer OIC CTrainee aelf-eitlraate ^ 3 point 
scale) 

<63 Ability to Perform Desired Job (Trainee self^estlaate 3 Paint scal^) 

<7) Expacted Start lug Salary (liourl;y) 

<8) Expected Top Salary Chourly) 

C9) Least Pay Acceptable (hourly) 

(10) Job Motivation (7 item scala score) 

(11) dumber of Relevant Ways to Search foT Job 

(12) Number of Importarit rhlngs to mk Job Intervtewer 

(13) VHiat to Do Beyond First Job (If Re levant responses) 

(14) Long Term Job Plans (Status level of job choice) 

(15) Hoi^ to Carry Out Plang (// Relevant reiponsea) 

(16) P€&r Rating (6 item scale score) 

(17) Get Along with Family (3 point scale) 

<18) Faiaily Feeling About XralTiee—irralnee estimate of ho%j faclly thinks 
he/slie is doing (3 point scale) 

CIS) Financial AaalgtMce to Family (Amomt of mVBTf $10 earned billing 
to cDntrlbute once job le olCalned) 

(20) Save Money CNo/yas) 
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(21) Number of People or Ins tttmclcas In the Cnmmunitv ntvinE Trninee 

(22) Health Proble™ <No/Yes) 

(23) Amount of Trouble with Police (4 point scale) 

(24) Importance of Keeplni Out of Trculle (3 point scale) 

(25) NumbiT of Jobs Willing to Tratn fcr 

(26) Number of Differeiit Training krmB Changes while In OIC 
(2 7) Nuniber of Days i^aent (from cLassas) 

(28) Vocational Ifistructor latlng (10 Item scale score) 

(29) Counselor RatlTig (10 item scale gcpre) 

(30) Remedial Instructor Rating <10 Item scale score) 
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APPENDIX B 
Poat'-Prog ram Criterion VaTlableg 

(1) Number of Con tacts Cto loeata EK-Traliiee) 

(2) Working Now (at tine of interviaw - No/Yes) 

(3) Blue Collar /Wilte Collar (work sitting) 

(4) Job Quality C3-=itein scale) 

(5) Numbar of Hours WoTked per Week 

(6) Length of Stay on Prag^nt Job 

(7) Number of Jobs Held slitce OIC 

(8) Numbar of Sourcaa used to find Firat Job 

(9) Numbar of Places Intervlewad Before First Jeb 
CIO) Numbar of applications Filed 

CU) EKCent to i^hlch rrainee Met EKpactatlons C3-point scale) 
Cl2) Amount of Starting Salary (Present or moat recently held job) 

(13) Pay Raise (No/Yes) 

(14) Promotion (No/Yes) 

(15) Job Satisfactlori (9-ltem scale score) 

(16) Top Salary E^cpectatlon 

(17) Laval of Job Sought (If left or lost prasent Jot) 

(18) Job Knowledge (TraiTiee estimate - 3-poliit scale) 

(19) Ways to Search for Nesct Job (# relev^ant responiei) 

(20) Laval of Long Range Job Plans 

(21) Ability to Achieve Plans (if relevant responses) 

(22) Going to School TIpw (No/Yes) 
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(23) Get Along with Family {3*point scale) 

(24) Family Feelings about how Trainee la doing (Trainee Estimate-* 

scale) 

(25) Financial Contribution to Family (# of dollais for each $10 
eaTned) 

(26) ^iu^lbe^ of People Giving Trainee "Hard Time-* 

(27) Health Problems (No/Yes) 

(2 8) Wumber of times seen a Doctor (since leaving OIC) 

(29) Save Money (3-polnt scale) 

(30) Importance of Keeping Out of Trouble (3-pQint scale) 

(31) Use of Cradlt (3-point scale) 

(32) Ease of Obtaining Credit (3-point scale) 

(33) Numbsr of Visits to State Emplo^mient Service (sirice QIC) 

(34) Uneniployaent Payments (No/Yes) 

(35) Employer Rating C4-iteffl scale score) 
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APPENDIX C 



Twenty-Nini CitlM 


from which OIC Proiects 


Were Dtmm Sot 


Study PartlGlpation 


Albuquerque J Wew Mexico 


Mllwaukeij WiscQiisiri 


Atlanta, Georgia 


Minneapolis s MltiMiota 


Boston J Massachusetts 


Montgonteyjr p Alabaina 


Brooklyn j New York 


Oklahoma City, OklahoBia 


Chicago^ Illinois 


Philadelphia, Pennayl'vanla 


Clevelarids Ohio 


PhoanlK^ Arlzoaa 


Detroit, Michigan 


Providen^i, ^ode Island 


Eries Petmaylvania 


Roanoke iVlrglftia 


HunCingtonj Wast Virginia 


Rocky MoTOt^ North Carolina 


Indianapolis, Indiana 


Spokane j ¥ashliiiton 


Jackson , Miss Isi ippl 


Stockton 3, Calif crnla 


Ki7Dxvllle, Tennessee 


Topekaj KmBBB 


Little Rock j Arkansas 


l^aehlngton-^ p. C* 


Lubbock i Texas 


VJilmlngconp Dela^^are 


Menlo Park, California 
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